Complex event processing (CEP) is designed to continuously process incoming events in a push-based manner in order to provide results for given queries. These continuous queries are usually deployed to a system, and users may subscribe to their results. While the queries are typically designed to meet the requirements of the users, the users' satisfaction may decrease because of factors such as, for example, changes in user preferences, the characteristics of the input streams, etc. Adapting the system to the user's modified requirements usually involves starting new queries, which are often modified versions of the old ones. Depending on the skills of the user, for example, this may even require involving information technology staff or persons with specific or deep knowledge of the processing and/or business environment to help build, deploy, and/or initiate newly developed queries.
CEP is a relatively new field, but there are several commercially available techniques that relate to query formulation and result presentation. These currently available CEP query formulation and result presentation techniques typically borrow ideas from the Database Management System (DBMS) field.
Database management systems generally execute queries once and usually in a rather short time window using a pull-based approach. If the user is not satisfied with the results, the user may modify the query and execute it again. Graphical user interfaces (GUIs) exist in order to help enable users to adjust parameters of such queries. Changing the settings of the queries via GUIs or the like allows a query to be modified before it is executed again. Unfortunately, however, this approach involves changing parameters between subsequent executions of separate, ad-hoc queries. It thus will be appreciated that there is no ability to modify continuous CEP queries during runtime. Furthermore, it will be appreciated that the crafting of new queries may require specific or deep knowledge of the processing system, business environment, etc.
As DBMS queries are not continuous, there conventionally has been no need to modify them at runtime. Nevertheless, basic ideas for doing so have been proposed in the past. Such proposals have generally involved merely providing the possibility for presentation of early results (e.g., such as those results that are available prior to completion of query execution) and providing the possibility to abort the execution of a query in progress in the event that the at least some results indicate that yet further results would be of little or no additional use or value.
Conventional CEP engines, which as noted above borrow ideas from the DBMS field, typically support the formulation of queries using a query language (such as, for example, SQL) and/or some graphical query building tool. Depending on user skills and the application domain, for example, this query formulation may be performed by a business user who also will use the query results. In some cases, the user may require assistance from others. After the formulation of queries has been completed, the prepared queries can be executed.
Graphical user interfaces that present the results of CEP queries to users sometimes allow for modifying the way that the results are presented or the way that one or more portions of the results are shown. As such, results are collected and a view of them is prepared while the original query stays unchanged within the CEP engine. And although research has been conducted on how the resource consumption of CEP engines can be measured and predicted, the risk of a CEP engine running out of resources while executing a set of queries remains.
DBMS query modification techniques—and many CEP techniques based on them—unfortunately suffer from disadvantages that DBMS have with respect to CEP engines in the domain of continuous queries over data streams.
Furthermore, with respect to current CEP systems, if the user is no longer satisfied with a query within a CEP engine, the user will have to stop it and start an updated version of the query. As alluded to above, this may require assistance from other people, as the user may not be familiar with the applicable query formulation techniques. Moreover, stopping a query and starting an alternative one may cause downtime, as executing new CEP queries likely will take some time before being able to provide new or updated results in many cases. And although potential downtime can be reduced by continuing the execution of the old query while the new one is being initialized, it will be appreciated that this approach will typically involve additional resources for the parallel execution of both versions of the query.
While substantially reducing the risk of a system overload, techniques developed for adapting CEP queries at runtime with respect to their resource usage have not been directed to improving the user experience by adapting query results with respect to a user's preferences.
Indeed, approaches that allow the user to dynamically configure result presentations (e.g., by selecting a portion of buffered query results for presentation) do not influence the query execution within the system. In order to be able to provide a wide spectrum of possible results, they have to continuously derive many results that afterwards may be discarded in whole or in part by the presentation layer. This approach involves substantial overhead and potentially a considerable amount of waste with respect to resource usage of the CEP engine.
Thus, it will be appreciated that there is a need in the art for improved techniques for preparing, executing, modifying, and/or visualizing queries in a CEP environment where data continuously or substantially continuously streams in.
One aspect of certain example embodiments relates to the ability to modify continuous complex event processing (CEP) queries during runtime. In certain example embodiments, the results of the queries are presented in a graphical user interface (GUI) manipulable by a user, where such manipulations may (1) provide an indication as to whether and which parameters may be of interest, (2) enable query customization based on such parameters, and/or (3) affect the execution of the queries. In cases where the execution of the queries is affected, modifications may be made in certain example embodiments without having to restart a query and/or without having to run different versions of a query or different queries in parallel.
Another aspect of certain example embodiments relates to allowing for the adjustment of already-initiated queries during runtime in order to help continuously or substantially continuously meet a user's requirements. In certain example embodiments, parameters of a query may be adapted dynamically, e.g., without the need to redeploy it. Adaptable parameters may in certain example instances be determined automatically from the original query.
Another aspect of certain example embodiments relates to gathering information regarding how to adjust parameters of a CEP query based on, for example, direct user feedback or interaction with a user interface, monitoring of user behavior, etc. Elements of a user interface according to certain example embodiments (e.g., a GUI) may help allow the user to provide concrete instructions or at least “hints” as to how the output of the query may or should be changed.
Another aspect of certain example embodiments relates to collecting information about how the user actually uses an interface according to certain example embodiments. Based on this or other information (e.g., information described herein as being collected or derived), certain example embodiments may further derive appropriate changes to the system parameters that may adapt the future query behavior.
Still another aspect of certain example embodiments relates to adapting CEP queries at runtime to help improve the user experience by adapting query results with respect to the user's preferences, influencing the CEP queries themselves (e.g., provided that resources are available), and/or helping to manage the utilization of resources.
Yet another aspect of certain example embodiments relates to modifying the result of a computational process rather than the result presentation, which may in certain example embodiments help conserve resources.
In certain example embodiments, a method of executing queries in a complex event processing (CEP) environment is provided. A stream of continuous event data is received. A query to be run on the received stream is identified, with the query including at least one adaptable parameter. The identified query is executed. Results of the identified query are displayed in a user interface. Via the user interface, input indicative of a change to the adaptable parameter is received. The identified query is modified based on the received input, with the query being dynamically modified during runtime and as the stream of continuous event data is being received.
In certain example embodiments, a method of configuring a system for executing queries in a complex event processing (CEP) environment is provided. User input defining at least one continuous query to be run on at least one stream of event data is received. The at least one query is parsed so as to (a) identify at least one adaptable parameter within the at least one query and (b) develop a logical query plan or graph based on the user input defining the query, with the logical query plan or graph describing at a high level steps to be performed when the query is run. The logical query plan or graph is translated into a physical query plan or graph, with the physical query plan or graph including executable logic to be used in the execution of the query. In connection with the translating of the logical query plan or graph into the physical query plan or graph, access points enabling a user to dynamically modify the at least one adaptable parameter of the at least one query at runtime without interrupting execution of the at least one query and while continuing to receive the at least one stream of event data are registered.
In certain example embodiments, a method of executing a query in a complex event processing (CEP) environment is provided. A system may be configured in accordance with the above-described or another method. The identified query is executed. Results of the identified query are displayed in a user interface. The query is updated based on (a) user input received via a user interface indicative of a change to the at least one adaptable parameter directly, and/or (b) a way in which the user interacts with the user interface. The updating is performed without redeploying the query.
In certain example embodiments, there are provided non-transitory computer readable storage mediums tangibly storing instructions that, when executed by at least one processor of a system, perform the above-described and/or other methods.
In certain example embodiments, there are provided systems for implementing the above-described and/or other methods. For instance, in certain example embodiments, a continuous event processing (CEP) system is provided. An adapter is configured to receive data from at least one event stream. A query design tool is configured to receive user input corresponding to at least one query executable on at least one said event stream. A query parser is configured to (a) identify at least one adaptable parameter within the at least one query and (b) develop a corresponding logical query plan or graph based on the at least one query, with each said corresponding logical query plan or graph specifying steps to be performed when the query is run. A query translator is configured to translate each said logical query plan or graph into a corresponding physical query plan or graph, with each said physical query plan or graph including executable logic to be used in the execution of the query. A register of access points enables a user to dynamically modify the at least one adaptable parameter of the at least one query at runtime without having to redeploy the at least one query and while continuing to receive the at least one stream of event data. At least one processor is configured to execute the at least one query on the at least one stream based on the physical query plan or graph. A user interface is configured to display results of the query executed by the at least one processor, with the user interface including one or more elements enabling the user to adjust the at least one adaptable parameter using the register of access points.
These aspects and example embodiments may be used separately and/or applied in various combinations to achieve yet further embodiments of this invention.